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'i B STB ACT 

The system discussed (in this paper was successfully 
used to track approximately 17,000 students participating in various 
educational programs and to measurie their gains xhtcugh achievement 
test results. It was developed for a school district with twelve 
anppleaentary instruction programs, and has been in use since the 
.1975-76 school year- The evaluation method was used to fulfill 
rdEederal reporting requirements and to conduct longitudinal studies in 
■easuring the impact of instructional programs. Programs xn which 
Jdndergax ten through high school students were tracked included: 
Title I, Title VII, State Bilingual, Bigrar.t. and Experimental 
'Schools. Inservice training was conducted to familiarize teachers and 
coQSselozs with the materials require^ in the process. Greater speed 

reporting test results, and increafeed specificity of scores for 
each student helped to alleviate teacjier resistance vo the extra 
tecozd keeping involved. Appendices itclude a list of references, 
ezaaples of scoring sheets, program code numbers, a list of the 
scoring codes, their corresponding programs, and the matching design. 
lESl 
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ABSTRACT 

This system is a method for setting up a testing program 
that will allow evalwators <"nd researchers to track students 
over periods of time, thus allowing for the measurement of 
sustained effect. The ^^ystem is both economical and easy to 
implement* The system was developed for a school district 
with 12 supplement ary instructional programs. This process 
also provides a structure for comparing students in various 
programs using test data, and an efficient way for collecting 
data needed to complete the required federal reports (Title I, 
Migrant, etc) . 
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The purpose of this paper is to present a systc^m, which was success- 
fxilly used to track students participating in various educational pro- 
grams and to measure their gains through achievement test results - 

INTRODUCTION 

Educational program evaluation was officially sanctioned with the 
passage by Congress of the Elementary and Secondary Education Act (Title I) 
in 1965. The purpose of evaluation, as introduced by ilobert Kennedy, 
was for accoxmtability. In the last 15 years, evaluation has evolved con- 
siderably. Originally dominated by methods from experimental psychology, 
evaluation has become interdisciplinary and has incorporated methods from 
the behavorlal and social sciences. As Congress has allocated more money 
to education, it has also further defined and required program evaluation. 
The Education Division General Administrative Regulations (EDGAR, 1980) state: 

" A grantee shall evaluate at least annually — 

(a) The grantee's progress in achieving the objectives in its 
approved application; 

(b) The effectiven-iss of the project in meeting the purposes 
of the program; and 

(c) The effect of the project on persons being served by the 
project. . 

As evaluation has evolved, so has conceim over the longitudinal effective- 
ness of various educational interventions over several years (Kirst & Jung, 
1980; Tallmadge, 1976). Numerous studies indicating the negative impact of 
varioxis educational programs have increased societies' concern over the effect- 
iveness of edxication (Coleman Report, AIR Report on Bilingual Education, etc). 

The basic method to measure the success or failure, of a program has been 

achievement tests. 
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This paper describes a system that has successfully been used in 
tracking student achievement over several years. It was originally 
developed for evaluating a school district's Title I and Experimental 
Schools Programs. It was also used to compare Title I, Title VII, ESAA, 
State Bilingual, Migrant and SCE Programs. This system easily fulfills 
the sustained effect requirements of the Title I regulations. For schools 
wishing to use this system, an assxmiption is made that implementation 
evaluation has already taken place before test data is used to measure 
orogram impact. Approximately 17,000 students were tracked using this 
system, 

STUDENT TRACKING SYSTEM MODEL 

The student tracking system was designed to provide test results 
for use in completing required federal program evaliiation reports. It 
also allows for the ccmparison of various instructional programs using 
test data. In this system, each student is assigned a xinique nine digit 
ID number. A description of nine digits in the number is: 

never 

changes^ changes 

ABC D 
00/9/04/1245 

The numbers in section A indicate the assigned program code. For 
example, a Title I Reading Program may be assigned a program code number 
of 10. A Title I Math Program may be assigned a program code number of 
II. If it is possible for the same student to be in both programs, the 
combined program code number could be J3. The 00 in column A would be 
changed to reflect the instructional program in which students participate. 

The number under B indicates the year that the student entered the 
tracking system. Once this number is assigned, it- never changes. During 
the 1978-79 school year, this number woiild have been S; during the 1979-80 
school year, it was 9; during the 1980-81 school year, it is 0. Column 
B allows one to determine Low many years of test data should be on file 
for a student. These numbers are never changed. 



The ninnber in colunm C indicates the grade that the student is in 
«faen entering the trackdLng system. Once this number is assigned, it 
never changes. 

The numbers in column D consist of unique computer generated 
numbers to assure no duplication. Numbers under D allow for 9999 stu- 
dents to enter the tracking system at the same time. No two students 
would have the same ID numbers. However, all could have the same 
numbers in columns A, B and C. 

A sample of support programs for which the system was used are: 
INTERVAL PROGRAM CODE PROGRAM 



10-19 



00 
10 
II 
12 
13 
14 
15 
16 



Not in any special program 
Title I Oral Langxiage 
Title I Reading 
Title I Aides 

Title I, both Reading & Aides 
Migrant Conmunication Skills 
Migrant Math 
Migrant both C.S. & Math 



20-29 



r.o 

21 
22 
23 



ESAA Bilingual 

£SAA Oral Language 

ESAA Cultural Arts 

ESAA, both Oral Language & 

Cultural Arts 



30-39 



35 
36 
37 



30 
31 



32 
33 
34 



State Bilingxial 

State Bilingual & Title I 

Oral Langiiage 

State Bilingual & Title I Reading 
State Bilingual & Title I Aide 
State Bilingual & Title I Reading 
Aides 

State Bilingual & Migrant 
State Bilingual & ESAA 

State Bilingual & ESAA Oral 

Language 

State Bilingual & ESAA Cultural 
Arts 

State Bilingual & ESAA Oral Langus^e 
& Cultural Arts 



ERLC 
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As an example, a stiident has an ID number of 396001212, This stu- 
dent Is in the State Bilingual Program and has also participated in the 
Emergency School Aide Act (ESAA) Program in Oral Language Development and 
Cultural Arts Enrichment. The student entered the tracking system during 
the 1976-77 school year and was in kindergarten, Dtiring the current school 
year, 1980-81, the student should be in the 4th grade. This should be the 
fifth year that the student has been tested. 

This system has been used to complete the Title I Annual Evaluation 
Report in the following manner. At the beginning of the school year, ID 
numbers were printed on Cal-stik labels by the Education Service Center, 
Region 20, Data Processing Center. Inservice was conducted with all the 
counselors to explain the tracking system and their roles. The counselors 
held an inservice with the classroom teachers and distributed the labels. 
The classroom teachers affixed a label to each student's Permanent Record 
Card (PRC). Using the PRC's, each teacher encoded the appropriate ID 
number on the pretest student answer sheet. It was not necessary to assign 
program codes for the pretest, so the first two digits were 00. All coding 
was done during the week of- testing Cthe actual testing was done on Tuesday, 
Wednesday and Thursday). The same procedures wers used for the posttest, 
only the 00 was replaced by an appropriate program code. 

The test instruments were scored by the publisher's scoring center.* 
The center was instrtxcted to merge the Title I codes and provide a computer 
printout of pre and posttest scores for ich student by grade level. The 
reports included a frequency distribution of GE gains for each subtest. The 
gains were then converted into tenths-of-a-month by the school district eval- 
uator in order to comply with the Tit^ I Annual Evaluation Report format. 
In previous years, it had taken several weeks to compile and complete the 
section of the report concerning test scores. Using this system, it took 
approximately one-half hour per grade level. 

* Since the development of this system, the scorixig contract has been trans- 
ferred to ESC, Region 20 which also has the software for Implementing this 
student tracking system. 



The tracking sy:»tem was easily implemented at the elementary school 
level. The teachers maintained the PRC's and therefore had easy access 
to the ID numbers* At the secondary level, the PRC's were kept in the 
counselors* office and were not easily accessible. Also, there was more 
teacher resistance toward the Increased work of encoding ID numbers on 
their students' answer sheets. However, when the teachers saw the print- 
outs and how useful the system was, this resistance was alleviated. 

A second problem was the assignment of ID numbers. At first, several 
persons were responsible for assigning ID nxxmbers. The result was the same 
number being assigned to two different students* This was resolved by 
having the numbers generated by computer and printed on labels. 

The student tracking system has been in use since the 1975-76 school 
year. Although originally designed to provide data for completing the 
annual Title I Evaluation, it has been used to compare all federal pro- 
grams and LEA programs. It is also currently being used to collect data 
to meet the Title I sustained effects requirement. 

EDUCATIONAL SIGNIFICANCE 

Most federal programs require the reporting of achievement test 
results in the annual evaluation reports. The Title I requirements 
include a sustaining effect report. This tracking system allows school 
districts to fulfill these reporting requirements and to conduct longi- 
tudinal studies using achievement test data in measuring the impact of 
Instructional programs. 
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APPENDIX 



PRE/POST FREQUENCY DISTRIBUTION 
PROGRAM CODE 
LETTER TO SCORING CENTER 



10 



CLASS RECORD SHEET 



IX 
1/^ 



itl 



» MAM EQUIVAUm 



12 



Gfc 



Ui^. 



b 1 


> 




0 0 


0 0 




3 1 


2 3 




1 4 


1 1 




1 2 


0 2 




1 0 


0 0 




0 0 


4 0 




1 0 


i: 1 




1 0 


0 1 




y i 


1 ^ 




0 3 


J 1 




7 1 


1 0 




2 5 


2 0 




0 2 


0 3 




2 ^ 


1 2 




3 2 


0 1 




0 0 


2 0 




0 0 


3 2 




3 6 


0 1 




2 1 


0 0 




u a 


0 J 


0 ' 
1 


1 0 


0 0 




1 u 






1 0 








II 0 







12' 



ERIC 



PROGRAM 

OO, , . • • - All Students- v^^o ARE NOT in Programs 

lO^ , titl^ ^ 

. Titl« 'I student attending Reading Lab 

1?^ , Title I Migrant 

^0. .-'•••SSAA bilingual 

n -'•••HSAA ^ Title I (K-1 ONLY) 
.'••••£SAA 5 Migrant (K-1 ONLY) 

^4 , ' • * * "BSAA ^^sic 

2S ,-*'*'£SAA Pilot 

^6, '••^•Bdgev'ood High School Special Program 

3o .Stat^ Compensatory Education 

^0 . • • • • -State Bilingual 

prograM_combinati^^ 

Si .•••••Sta.t^ Bilingual, Tiae I 

" -S^* -.State Bilingual, Title I attending 
- Reading Lab 

S5 -State Bilingual, Migrant 

S4- State Bilingual, Title^I, ESAA u^u^^ 
SS" State Bilingual and ESAA 

^6 -State Bilingual, M^SJ^^^' ^f^A y ^ ^^i^j, 

State Bilingual, state Compensatory tauc 

4q . -PlaP A especial Education) 

S8__,.----.Sta'^« Bilingual, plan A 
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INDEPENDENT SCHOOL DISTRICT 



March 15, 1979 



Ms. Rhonda Dillon 

CTB/McGraw-Hill 

Customer Service 

Del Monte Research Park 

Monterey, CA 93940 

Dear Rhonda: 

- a description of the scoring services which we 

r ui^:' ^luhouT-e format of Jhis js different, the 
services are the same as those - ecexved last year. 

I. T.ist of Sch ^^T^^t^^ades Tested; 

(See attached list of schools.) 

II. S pecial Codes 

A. columns A & B contain 2-pigit Program codes. 

Column ■ Bubble 
Program — 



A 
B 



0- 

0 



1 . All Students who are not in 
programs listed below 

A - 1 

2. Title I J _ 0 

3. Title I Student attending B - ^ 
Rieading Lab. 

A - 1 

4. Title I Migrant » - 2 



A - 2 

5. ESAA Bil: igual J _ 0 



A" - 2 



6. ESAA and Title I J I 1 



A - 2 

7. ESAA and Migrant g _ 2 

A - 2 

8. ESAA Basic g _ 4 

A - 2 

9. ESAA Pilot • B _ 5 



Rhonda Dillon 
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March 15, 1979 



Program 



Column Bubble 



10. Edgewood High School A - ^ 
Special Program ^ 

11. State Compensatory A - 

B - u 

12. State Bilingual 



13. State Bilingual and 



14. State Bilingual & Title I 
Reading Lab 

15. State Bilingual & Migrant 



19. State Bilingual and State 
Compensatory 



A - 5 

B - 0 



A r 5 



Title I ® 



I 



A - 5 

B - 2 

A - 5 

B - - 3 



16. State Bilingual, Title I A - 5 
and ESAA ^ 

17. State Bilinr-ial and ESAA A - | 

B - -* 

18. State Bilingual, Migrant A - 
and ESAA ^ 

A - 5 

B - 7 



B. columns C-I or C-J contain 7-digit 

student TO number. All pre-post matching will be b> 
these 7-digit ID numbers, beginning with Column C 
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March 15, 1979 



III, Matching Design 

Pre-Test 
Fall, 1978 

Grade Level Batch # Matched To 



K 


A 


A6A0 


I 


B 


AoAl 


2 


C 


A6A2 


3 


1 


A6A3 


4 


1 


A6A4 


5 


2 


A6A5 


6 


2 


A6A6 


7 


3 


A6A7 


8 


3 


A6A8 


9 


4 


A6A9 


10 


4 


A6AB 


11 


4 


A6AA 



Post-Test 
Soring. 1978 



Grade 



K 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 



Level 

A 
B 
C 
1 

• 1 
2 
2 
3 
3 
4 
4 
4 



Pre/Post Combined Code Matches 

Grades 



00 

10, 21, 51, 54 

11, 52 

12, 22, 53, 56 

20, 21, 22, 54, 55, 56 

24 

25 

30 

50, 51, 52, 53,54,55,56,57 



K-8 

K-6 

3-6 

1-11 

K-4 

7-8 

9-10 

1-8 

K-6 



Program 

Other 
Title I 
Reading Lab 
Migrant 
ESAA Bilingual 
ESAA Basic 
ESAA Pilot 
SCE 

State Bilingual 



We will need the 2 digit codes that cover the listed programs 
merged, and a PPMCRS for each program. 

Sincerely, 

Al Noonan 
Evaluator 



AN:x3as 



Attachment 



